Cache Conscious Column Organization in In-Memory Column Stores

Cost models are an essential part of database systems, as they are the basis of query performance optimization. Based on predictions made by cost models, the fastest query execution plan can be chosen and executed or algorithms can be tuned and optimized. In-memory databases shift the focus from disk to main memory accesses and CPU costs, compared to disk based systems where input and output costs dominate the overall costs and other processing costs are often neglected. However, modeling memory accesses is fundamentally different and common models do not apply anymore. This work presents a detailed parameter evaluation for the plan operators scan with equality selection, scan with range selection, positional lookup and insert in in-memory column stores. Based on this evaluation, we develop a cost model based on cache misses for estimating the runtime of the considered plan operators using different data structures. We consider uncompressed columns, bit compressed and dictionary encoded columns with sorted and unsorted dictionaries. Furthermore, we discuss tree indices on the columns and dictionaries. Finally, we consider partitioned columns consisting of one partition with a sorted and one with an unsorted dictionary. New values are inserted in the unsorted dictionary partition and moved periodically by a merge process to the sorted partition. We propose an efficient merge algorithm, supporting the update performance required to run enterprise applications on read-optimized databases and provide a memory traffic based cost model for the merge process.